QUESTION ANSWERING

MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension (D19-58)

Challenge:	Factoid question answering systems extract answers for a question from passages, which are usually short spans of text . but, these spans would result in an unnatural reading experience in a conversational system . a pointer generator based full-length answer generator can be used with most QA systems .
Approach:	They propose a pointer generator based full-length answer generator which can be used with most QA systems.
Outcome:	The proposed system generates full length answer without relying on passage from which it was extracted.

Copied to clipboard

Adam Fisch, Alon Talmor, Robin Jia, Minjoon Seo, Eunsol Choi, Danqi Chen

Reading Comprehension as Natural Language Inference:A Semantic Analysis (2020.starsem-1)

Challenge:	MRQA datasets have been used to benchmark progress in general-purpose language understanding.
Approach:	They propose to combine 18 question answering datasets into one shared task to evaluate their generalization capabilities.
Outcome:	The best system achieved an average F1 score of 72.5 on the 12 held-out datasets, 10.7 absolute points higher than baseline based on BERT.

Copied to clipboard

Anshuman Mishra, Dhruvesh Patel, Aparna Vijayakumar, Xiang Li, Pavan Kapanipathi, Kartik Talamadupula

Practical Semantic Parsing for Spoken Language Understanding (N19-2)

Challenge:	In recent past, Natural language Inference (NLI) has gained significant attention, but its true impact has not been well studied.
Approach:	They propose to transform a large RACE dataset into an NLI model and compare it to a state-of-the-art model.
Outcome:	The proposed model outperforms the previous model on a question-answer concatenation form and a coherent entailment form.

Copied to clipboard

Marco Damonte, Rahul Goel, Tagyoung Chung

NeuralQA: A Usable Library for Question Answering (Contextual Query Expansion + BERT) on Large Datasets (2020.emnlp-demos)

Challenge:	Existing systems that can handle a user's utterance are unable to handle Q&A or SLU.
Approach:	They build a transfer learning framework for executable semantic parsing . they show it is effective for Q&A and for spoken language understanding .
Outcome:	The proposed framework is effective for Q&A and Spoken Language Understanding . it can be learned by exploiting data on other domains, the authors show .

Copied to clipboard

Victor Dibia

UKP-SQuARE v2: Explainability and Adversarial Attacks for Trustworthy QA (2022.aacl-demo)

Challenge:	Existing tools for Question Answering (QA) have challenges that limit their use in practice.
Approach:	They propose a library that integrates with existing infrastructure and offers helpful defaults for QA subtasks.
Outcome:	NeuralQA integrates well with existing infrastructure and offers helpful defaults for QA subtasks.

Copied to clipboard

Rachneet Sachdeva, Haritz Puerto, Tim Baumgärtner, Sewin Tariverdian, Hao Zhang, Kexin Wang, Hossain Shaikh Saadi, Leonardo F. R. Ribeiro, Iryna Gurevych

PrimeQA: The Prime Repository for State-of-the-Art Multilingual Question Answering Research and Development (2023.acl-demo)

Challenge:	Question Answering (QA) systems rely on deep neural networks, which are difficult to interpret by humans.
Approach:	They propose an interpretable model that provides an explanation infrastructure for comparing models based on saliency maps and graph-based explanations.
Outcome:	The proposed methods can be used to compare models based on saliency maps and graph-based explanations.

Copied to clipboard

Avi Sil, Jaydeep Sen, Bhavani Iyer, Martin Franz, Kshitij Fadnis, Mihaela Bornea, Sara Rosenthal, Scott McCarley, Rong Zhang, Vishwajeet Kumar, Yulong Li, Md Arafat Sultan, Riyaz Bhat, Juergen Bross, Radu Florian, Salim Roukos

Jack the Reader – A Machine Reading Framework (P18-4)

Challenge:	Question Answering (QA) is a major area of research in Natural Language Processing (NLP)
Approach:	They propose a one-stop and open-source QA repository for question answering . it supports core QA functionalities like retrieval and reading comprehension . they say it will facilitate easy replication of state-of-the-art (SOTA) QA methods .
Outcome:	The proposed framework enables easy replication of state-of-the-art (SOTA) QA methods.

Copied to clipboard

Dirk Weissenborn, Pasquale Minervini, Isabelle Augenstein, Johannes Welbl, Tim Rocktäschel, Matko Bošnjak, Jeff Mitchell, Thomas Demeester, Tim Dettmers, Pontus Stenetorp, Sebastian Riedel

Contrastive Data and Learning for Natural Language Processing (2022.naacl-tutorials)

Challenge:	Many Machine Reading and Natural Language Understanding tasks require reading supporting text in order to answer questions.
Approach:	They propose a framework for Machine Reading that allows for quick prototyping by component reuse and evaluation of new models on existing datasets.
Outcome:	The proposed framework supports question answering, natural language inference and link prediction tasks.

Copied to clipboard

Rui Zhang, Yangfeng Ji, Yue Zhang, Rebecca J. Passonneau

MoEMoE: Question Guided Dense and Scalable Sparse Mixture-of-Expert for Multi-source Multi-modal Answering (2025.naacl-industry)

Challenge:	Current NLP models heavily rely on effective representation learning algorithms.
Approach:	This tutorial introduces contrastive learning and provides an introduction to the techniques.
Outcome:	This tutorial provides an introduction to the fundamentals of contrastive learning approaches and the theory behind them.

Copied to clipboard

Vinay Kumar Verma, Shreyas Sunil Kulkarni, Happy Mittal, Deepak Gupta

Identifying Supporting Facts for Multi-hop Question Answering with Document Graph Networks (D19-53)

Challenge:	Question Answering (QA) and Visual Question Answers (VQA) are well-studied problems in the language and vision domain.
Approach:	They propose a question-answer generation framework that learns attention across multiple sources and decodes this information for robust and unbiased answer generation.
Outcome:	The proposed framework can handle thousands of question types and scale to scale.

Copied to clipboard

Mokanarangan Thayaparan, Marco Valentino, Viktor Schlegel, André Freitas

Audio Query Handling System with Integrated Expert Models and Contextual Understanding (2025.emnlp-industry)

Challenge:	Recent advances in reading comprehension have resulted in models that surpass human performance when the answer is contained in a single, continuous passage of text.
Approach:	They propose a document-structured message passing architecture for the identification of supporting facts over a graph-structure based representation of text.
Outcome:	The proposed model outperforms a baseline reading comprehension test on raw text and shows that it is relevant for multi-hop reasoning.

Copied to clipboard

Naveen Vakada, Arvind Krishna Sridhar, Yinyi Guo, Erik Visser

Unsupervised Information Refinement Training of Large Language Models for Retrieval-Augmented Generation (2024.acl-long)

Challenge:	Existing chatbots are limited to specific audio tasks, but the domain of audio content related queries remains underexplored.
Approach:	They propose to use an intent classifier to route queries to audio-related experts using a diverse audio query dataset.
Outcome:	The proposed system outperforms state-of-the-art LLMs on custom audio tasks and MMAU sound set benchmarks.

Copied to clipboard

Shicheng Xu, Liang Pang, Mo Yu, Fandong Meng, Huawei Shen, Xueqi Cheng, Jie Zhou

A Pipeline for Generating, Annotating and Employing Synthetic Data for Real World Question Answering (2022.emnlp-demos)

Challenge:	Existing studies show that LLMs face challenges in effectively using retrieved information . authors propose a method that considers LLM as "Information Refiner"
Approach:	They propose a method that considers LLMs as "Information Refiners" they propose INFO-RAG, which is low-cost and general across various tasks .
Outcome:	The proposed method improves performance of LLaMA2 by 9.39% relative points . it is low-cost and general across various tasks, and is robust and in-context learning is possible .

Copied to clipboard

Matt Maufe, James Ravenscroft, Rob Procter, Maria Liakata

Self-Supervised Knowledge Triplet Learning for Zero-Shot Question Answering (2020.emnlp-main)

Challenge:	Question Answering (QA) is a growing area of research . state-of-the-art QA models struggle on out-of domain documents without fine-tuning .
Approach:	They propose a pipeline for validating and training QA data and an interface for human annotation.
Outcome:	The proposed pipeline improves QA performance on domain-specific datasets while preserving the accuracy of the model.

Copied to clipboard

Pratyay Banerjee, Chitta Baral

Out of Style: RAG’s Fragility to Linguistic Variation (2026.eacl-long)

Challenge:	Current supervised Question Answering methods rely on expensive data annotations and can introduce unintended annotator bias.
Approach:	They propose a self-supervised task over knowledge graphs that can be supervised by a data annotation tool.
Outcome:	The proposed task performs better than pre-trained language models on a large dataset.

Copied to clipboard

Tianyu Cao, Neel Bhandari, Akhila Yerukola, Akari Asai, Maarten Sap

Unveiling Divergent Inductive Biases of LLMs on Temporal Data (2024.naacl-short)

Challenge:	linguistic reformulations impact both retrieval and generation stages, leading to a relative performance drop of up to 40.41% for less formal queries and 38.86% for queries containing grammatical errors.
Approach:	They evaluate two retrieval models and nine LLMs across four QA datasets and examine how linguistic reformulations impact RAG performance.
Outcome:	The proposed models show that linguistic reformulations significantly impact both retrieval and generation stages, leading to a performance drop of up to 40.41% for less formal queries and 38.86% for queries containing grammatical errors.

Copied to clipboard

Sindhu Kishore, Hangfeng He

Generative Data Augmentation using LLMs improves Distributional Robustness in Question Answering (2024.eacl-srw)

Challenge:	Temporal relations play a crucial role across diverse applications, including event summarization, predicting future events and medical information processing.
Approach:	They evaluate the performance of large language models in the analysis of temporal data using two prompt types, Question Answering and Textual Entailment.
Outcome:	The proposed models show that they are biased towards specific temporal relationships, while GPT-3.5 prefers “AFTER” for implicit and explicit events, while TE models lean towards “BEFORE”.

Copied to clipboard

Arijit Chowdhury, Aman Chadha

Let Me Know What to Ask: Interrogative-Word-Aware Question Generation (D19-58)

Challenge:	Existing domain adaptation methods do not account for unseen natural distribution shifts.
Approach:	They perform experiments on 4 different datasets under varying amounts of distribution shift . they analyze how "in-the-wild" generation can help achieve domain generalization .
Outcome:	The proposed approach augments reading comprehension datasets with generated data to improve robustness towards natural distribution shifts.

Copied to clipboard

Junmo Kang, Haritz Puerto San Roman, Sung-Hyon Myaeng

D-NET: A Pre-Training and Fine-Tuning Framework for Improving the Generalization of Machine Reading Comprehension (D19-58)

Challenge:	Existing models focus on generating questions based on text and the answer to the generated question.
Approach:	They propose a pipelined system that predicts the type of interrogative word to be generated . they also propose qg models that can be used to generate questions based on text .
Outcome:	The proposed system improves on the task of QG in SQuAD, improving from 46.58 to 47.69 in BLEU-1, 17.55 to 18.53 in blu-4, 21.24 to 22.33 in METEOR, and 44.53 to 46.94 in ROUGE-L.

Copied to clipboard

Hongyu Li, Xiyuan Zhang, Yibing Liu, Yiming Zhang, Quan Wang, Xiangyang Zhou, Jing Liu, Hua Wu, Haifeng Wang

Empower Large Language Model to Perform Better on Industrial Domain-Specific Question Answering (2023.emnlp-industry)

Challenge:	MRC requires machines to understand text and answer questions about the text.
Approach:	They propose a simple system Baidu submitted for MRQA 2019 Shared Task that focused on generalization of machine reading comprehension (MRC) models.
Outcome:	The proposed system is ranked at top 1 of all participants in terms of averaged F1 score.

Copied to clipboard

Fangkai Yang, Pu Zhao, Zezhong Wang, Lu Wang, Bo Qiao, Jue Zhang, Mohit Garg, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang

AIT-QA: Question Answering Dataset over Complex Tables in the Airline Industry (2022.naacl-industry)

Challenge:	Large Language Models (LLMs) have gained popularity but lack specific domain knowledge in domain-specific tasks.
Approach:	They propose a model interaction paradigm that empowers LLM to achieve better performance on domain-specific tasks where it is not proficient.
Outcome:	The proposed approach outperforms the commonly used LLM with retrieval methods in domain-specific tasks.

Copied to clipboard

Yannis Katsis, Saneem Chemmengath, Vishwajeet Kumar, Samarth Bharadwaj, Mustafa Canim, Michael Glass, Alfio Gliozzo, Feifei Pan, Jaydeep Sen, Karthik Sankaranarayanan, Soumen Chakrabarti

Zero-shot Cross-lingual Transfer With Learned Projections Using Unlabeled Target-Language Data (2023.acl-short)

Challenge:	Table Question Answering (Table QA) systems have been shown to be highly accurate when trained and tested on open-domain datasets built on top of Wikipedia tables.
Approach:	They propose a domain-specific Table QA test dataset to test Table Question Answering systems on open-domain datasets built on top of Wikipedia tables.
Outcome:	The proposed methods are highly accurate when tested on open-domain datasets built on top of Wikipedia tables.

Copied to clipboard

Ujan Deb, Ridayesh Parab, Preethi Jyothi

Pre-Training Methods for Question Reranking (2024.eacl-short)

Challenge:	Zero-shot cross-lingual transfer is enabled by pairing the language adapter in the target language with an appropriate task adapter within a source language.
Approach:	They propose to use unlabeled text to enhance zero-shot transfer by pairing language adapters with task adapters in a target language.
Outcome:	The proposed framework improves on three cross-lingual tasks with up to 11% relative improvement in Named Entity Recognition (NER), Question Answering (QA) and Natural Language Inference (NLI).

Copied to clipboard

Stefano Campese, Ivano Lauriola, Alessandro Moschitti

Zero-shot Event Extraction via Transfer Learning: Challenges and Insights (2021.acl-short)

Challenge:	Existing methods for Question Answering to search for semantically similar questions are not suitable for new questions.
Approach:	They propose an unsupervised method for retrieving and ranking questions . they use a question retrieval model and a selection model to rerank questions based on their relevance .
Outcome:	The proposed method achieves state-of-the-art performance on QRC and Quora-match datasets . it provides better and cheaper access to answers than the system generated them .

Copied to clipboard

Qing Lyu, Hongming Zhang, Elior Sulem, Dan Roth

CNN for Text-Based Multiple Choice Question Answering (P18-2)

Challenge:	Existing methods for event extraction require expensive annotation and are not extensible to new event ontologies.
Approach:	They propose to use textual entailment and/or question answering queries to extract a zero-shot event from a set of TE and/ or QA queries.
Outcome:	The proposed method achieves acceptable results on ACE-2005 and ERE, but there is still a large gap from supervised approaches.

Copied to clipboard

Akshay Chaturvedi, Onkar Pandit, Utpal Garain

Source-Free Unsupervised Domain Adaptation for Question Answering via Prompt-Assisted Self-learning (2024.findings-naacl)

Challenge:	Existing models for text-based multiple choice question answering are based on a text.
Approach:	They propose a Convolutional Neural Network (CNN) model for text-based multiple choice question answering where questions are based on a particular article.
Outcome:	The proposed model outperforms several baseline models on the SciQ and TQA datasets.

Copied to clipboard

Maxwell Yin, Boyu Wang, Charles Ling

UKP-SQuARE v3: A Platform for Multi-Agent QA Research (2023.acl-demo)

Challenge:	Existing SFDA methods focus on the adaptation phase, overlooking the impact of source domain training on model generalizability.
Approach:	They propose a source-free domain adaptation approach for Question Answering where a model trained on a domain is adapted to unlabeled target domains without additional source data.
Outcome:	The proposed model outperforms existing methods in managing domain gaps and demonstrating greater stability across target domains.

Copied to clipboard

Haritz Puerto, Tim Baumgärtner, Rachneet Sachdeva, Haishuo Fang, Hao Zhang, Sewin Tariverdian, Kexin Wang, Iryna Gurevych

RAG4ITOps: A Supervised Fine-Tunable and Comprehensive RAG Framework for IT Operations and Maintenance (2024.emnlp-industry)

Challenge:	Current approaches to QA models are multi-dataset models, but combining expert agents can yield large performance gains over multi-agent models.
Approach:	They extend an online platform for QA research to support three families of multi-agent systems: agent selection, early-fusion of agents, and late-fusion.
Outcome:	The proposed model can be compared with multi-dataset models and achieve high inference speed and performance.

Copied to clipboard

Tianyang Zhang, Zhuoxuan Jiang, Shengguang Bai, Tianrui Zhang, Lin Lin, Yang Liu, Jiawei Ren

Syntactically Aware Neural Architectures for Definition Extraction (N18-2)

Challenge:	Large Language Models (LLMs) have improved the open-domain QA’s performance, but how to efficiently handle enterprise-exclusive corpora and build domain-specific QA systems are still not studied for industrial applications.
Approach:	They propose a general and comprehensive framework based on Retrieval Augmented Generation (RAG) and facilitate the whole business process of establishing QA systems for IT operations and maintenance.
Outcome:	The proposed framework achieves superior results on two kinds of QA tasks.

Copied to clipboard

Luis Espinosa-Anke, Steven Schockaert

Accurate Training of Web-based Question Answering Systems with Feedback from Ranked Users (2023.acl-industry)

Challenge:	Existing approaches to identify definitional knowledge in text corpora are based on Wikipedia-like definitions.
Approach:	They propose to combine Convolutional and Recurrent Neural Networks to train definitional knowledge in text corpora.
Outcome:	The proposed models can be applied to more noisy domain-specific corpora.

Copied to clipboard

Liang Wang, Ivano Lauriola, Alessandro Moschitti

Natural Context Drift Undermines the Natural Language Understanding of Large Language Models (2025.findings-emnlp)

Challenge:	Recent work shows that large-scale annotated datasets are essential for training state-of-the-art Question Answering (QA) models.
Approach:	They use large-scale annotated datasets to train question answering models . they use feedback data collected from deployed QA systems to provide cheaper supervision .
Outcome:	The proposed model improves on the large scale annotated datasets from QA systems . the proposed model can be easily supervised on large-scale unlabeled web data .

Copied to clipboard

Yulong Wu, Viktor Schlegel, Riza Batista-Navarro

Improving Pretraining Techniques for Code-Switched NLP (2023.acl-long)

Challenge:	generative Large Language Models (LLMs) are based on natural text evolution .
Approach:	They propose a framework for curating naturally evolved variants of reading passages from contemporary QA benchmarks and for analysing LLM performance across a range of semantic similarity scores.
Outcome:	The proposed framework evaluates QA datasets and LLMs with publicly available training data.

Copied to clipboard

Richeek Das, Sahasra Ranjan, Shreya Pathak, Preethi Jyothi

Self-Training Large Language Models for Tool-Use Without Demonstrations (2025.findings-naacl)

Challenge:	Multilingual pretraining models for code-switched inputs are a key component of NLP applications.
Approach:	They propose to use masked language modeling techniques to mask code-switched text that are cognizant of language boundaries prior to masking.
Outcome:	The proposed techniques improve performance on two downstream tasks, Question Answering (QA) and Sentiment Analysis (SA), compared to standard pretraining techniques.

Copied to clipboard

Ne Luo, Aryo Pradipta Gema, Xuanli He, Emile Van Krieken, Pietro Lesci, Pasquale Minervini

Beyond English: The Impact of Prompt Translation Strategies across Languages and Tasks in Multilingual LLMs (2025.findings-naacl)

Challenge:	Recent work augmented LLMs with tools to mitigate factual inaccuracies and computational errors.
Approach:	They propose a method to synthesise tool-use traces using the LLM itself.
Outcome:	The proposed method improves performance on a long-tail knowledge task, but not on other datasets.

Copied to clipboard

Itai Mondshine, Tzuf Paz-Argaman, Reut Tsarfaty

Multilinguality

Challenge:	Current LLMs are primarily trained on English data but also include data from other languages.
Approach:	They propose to use a pre-translation strategy to translate a task prompt into English before inference . they use 'a modular entity' that could be translated into four different languages .
Outcome:	The proposed strategies are based on a set of pre-trained data across 35 languages covering both low and high-resource languages.

Multi-Relational Question Answering from Narratives: Machine Reading and Reasoning in Simulated Worlds (P18-1)

Copied to clipboard

Igor Labutov, Bishan Yang, Anusha Prakash, Amos Azaria

Policy-based Reinforcement Learning for Generalisation in Interactive Text-based Environments (2023.eacl-main)

Challenge:	Question Answering (QA) has primarily focused on knowledge bases or free text as a source of knowledge.
Approach:	They propose a task of multi-relational QA over personal narrative using text worlds . they generate and release a lightweight Python-based framework for easily generating additional worlds and narrative .
Outcome:	The proposed framework combines elements of structured QA over knowledge bases and unstructured QA . it generates and analyzes five diverse datasets with dynamic narrative . the framework is lightweight and easy to use .

Copied to clipboard

Edan Toledo, Jan Buys, Jonathan Shock

Open Domain Question Answering with Conflicting Contexts (2025.findings-naacl)

Challenge:	Text-based environments allow RL agents to learn to converse and perform interactive tasks through natural language.
Approach:	They propose to switch from a value-based update method to a policy-based one within text-based environments and evaluate it on Coin Collector and Question Answering with interactive text (QAit).
Outcome:	The proposed policy-based agent is more generalised than value-based methods in two text-based environments designed to test zero-shot performance.

Copied to clipboard

Siyi Liu, Qiang Ning, Kishaloy Halder, Zheng Qi, Wei Xiao, Phu Mon Htut, Yi Zhang, Neha Anna John, Bonan Min, Yassine Benajiba, Dan Roth

Text Modular Networks: Learning to Decompose Tasks in the Language of Existing Models (2021.naacl-main)

Challenge:	Open domain question answering systems often rely on information retrieved from large collections of text to answer questions.
Approach:	They evaluate and benchmark three powerful Large Language Models with a dataset . they find that 25% of unambiguous open domain questions can lead to conflicting contexts .
Outcome:	The proposed model can't be used to answer questions with conflicting contexts . it can be fine tuned to provide richer information into the model's training .

Copied to clipboard

Tushar Khot, Daniel Khashabi, Kyle Richardson, Peter Clark, Ashish Sabharwal

Increasing Coverage and Precision of Textual Information in Multilingual Knowledge Graphs (2023.emnlp-main)

Challenge:	Existing approaches to decompose complex tasks into simpler ones do not require annotated decompositions.
Approach:	They propose a framework for building interpretable systems that learn to solve complex tasks by decomposing existing models into simpler ones solvable by existing models.
Outcome:	The proposed framework is more versatile than existing explainable systems for DROP and HotpotQA datasets, is more robust than state-of-the-art blackbox (uninterpretable) systems, and generates more understandable and trustworthy explanations compared to prior work.

Copied to clipboard

Simone Conia, Min Li, Daniel Lee, Umar Minhas, Ihab Ilyas, Yunyao Li

Which Works Best for Vietnamese? A Practical Study of Information Retrieval Methods across Domains (2026.findings-eacl)

Challenge:	Existing methods to generate knowledge graphs are unable to handle non-English textual information.
Approach:	They propose a task of automatic Knowledge Graph Completion to bridge the gap between English and non-English textual information.
Outcome:	The proposed method bridges the gap between the quantity and quality of textual information between English and non-English languages.

Copied to clipboard

Long S. T. Nguyen, Tho T. Quan

Robustness Evaluation of the German Extractive Question Answering Task (2025.coling-main)

Challenge:	Existing studies on Large Language Models (LLMs) are limited to single domains or curated datasets.
Approach:	They propose a domain-normalized, multi-domain benchmark for Vietnamese IR . they evaluate lexical, neural-sparse, late-interaction, dense, and hybrid paradigms .
Outcome:	The proposed benchmarks cover six domains and ten datasets across education, legal, healthcare, customer support, lifestyle reviews, and open-domain knowledge.

Copied to clipboard

Shalaka Satheesh, Katharina Beckh, Katrin Klug, Héctor Allende-Cid, Sebastian Houben, Teena Hassan

LLM as a Meta-Judge: Synthetic Data for NLP Evaluation Metric Validation (2026.acl-srw)

Challenge:	Existing evaluation benchmarks for Question Answering systems only include EM and F1 scores, but they overlook critical factors for the deployment of QA systems.
Approach:	They propose to define an evaluation method specifically tailored to the German language to evaluate the robustness of German QA models.
Outcome:	The proposed method extends existing methods to German language . it shows that all models are vulnerable to character-level perturbations .

Copied to clipboard

Lukáš Eigler, Jindřich Libovický, David Hurych

Challenge:	Existing evaluation metrics for natural language generation are expensive and time-consuming.
Approach:	They propose a framework that utilizes LLMs to generate synthetic evaluation datasets . they propose meta-correlation to measure alignment between metric rankings and human benchmarks based on synthetic data .
Outcome:	The proposed framework achieves meta-correlations exceeding 0.9 in multilingual QA and replaces human judgment with synthetic evaluation datasets.

COMPKE: Complex Question Answering under Knowledge Editing (2025.findings-acl)

Copied to clipboard

Keyuan Cheng, Zijian Kan, Zhuoran Zhang, Muhammad Asif Ali, Lijie Hu, Di Wang

Double Retrieval and Ranking for Accurate Question Answering (2023.findings-eacl)

Challenge:	Existing benchmarks for knowledge editing do not accurately evaluate how well models apply knowledge in real-life situations.
Approach:	They propose a benchmark to evaluate how well updated models apply new knowledge in real-life situations.
Outcome:	The proposed method achieves 39.47 accuracy on GPT-4o-mini but drops significantly to 3.83 on Qwen2.5-3B.

Copied to clipboard

Zeyu Zhang, Thuy Vu, Alessandro Moschitti

Think Twice: Measuring the Efficiency of Eliminating Prediction Shortcuts of Question Answering Models (2024.eacl-long)

Challenge:	Recent work shows that answer verification models can improve the state of the art in Question Answering . despite the fact that the supporting candidates are ranked only according to the relevancy with the question, the model still lacks the support needed for other answer candidates.
Approach:	They propose a double reranking model that selects the best support for each target answer . they propose 'second neural retrieval stage' to encode question and answer pair as query .
Outcome:	The proposed approach improves the state of the art in Question Answering . the proposed model ranked candidates according to relevancy and not the answer . but the proposed approach fails to provide the best support .

Copied to clipboard

Lukáš Mikula, Michal Štefánik, Marek Petrovič, Petr Sojka

FREB-TQA: A Fine-Grained Robustness Evaluation Benchmark for Table Question Answering (2024.naacl-long)

Challenge:	Existing work shows that Large Language Models (LLMs) are not robust to complex language understanding tasks due to reliance on spurious correlations of training datasets.
Approach:	They propose a method for measuring model reliance on spurious features by exploiting chosen biases on out-of-distribution (OOD) datasets.
Outcome:	The proposed method shows that the reported OOD gains of debiasing methods can't be explained by mitigated reliance on biased features, suggesting that biases are shared among different QA datasets.

Copied to clipboard

Wei Zhou, Mohsen Mesgar, Heike Adel, Annemarie Friedrich

Can Edge Probing Tests Reveal Linguistic Knowledge in QA Models? (2022.coling-1)

Challenge:	Existing benchmarks for Table Question Answering lack robustness . current systems generate inconsistent responses by performing coarse-grained changes in tables and questions.
Approach:	They formalize three major desiderata for a fine-grained evaluation of robustness of TQA systems . they aim to evaluate TQA responses on the content of relevant cells rather than on biases .
Outcome:	The proposed benchmarks show that state-of-the-art TQA systems fail in three aspects . the benchmarks are released publicly and are expected to improve over time .

Copied to clipboard

Sagnik Ray Choudhury, Nikita Bhutani, Isabelle Augenstein

Event Extraction as Question Generation and Answering (2023.acl-short)

Challenge:	grammatical knowledge is encoded in large pre-trained language models (LMs) this is done through supervised classification tasks to predict the grammamatical properties of a span using only the token representations coming from the LM encoder.
Approach:	They propose to use a supervised 'edge probing' task to detect grammatical knowledge in large pre-trained language models (LMs) this is done by encoding grammamatical properties using only token representations coming from the LM encoder.
Outcome:	The proposed model performs well when fine-tuned or in adversarial situations where the model is forced to learn wrong correlations.

Copied to clipboard

Di Lu, Shihao Ran, Joel Tetreault, Alejandro Jaimes

CMQA: A Dataset of Conditional Question Answering with Multiple-Span Answers (2022.coling-1)

Challenge:	Recent work on Event Extraction addresses the error propagation issue found in token-based classification approaches.
Approach:	They propose a Question Generation (QG) model that generates questions that leverage contextual information instead of fixed templates.
Outcome:	The proposed model outperforms all previous single-task-based models on the ACE05 English dataset.

Copied to clipboard

Yiming Ju, Weikang Wang, Yuanzhe Zhang, Suncong Zheng, Kang Liu, Jun Zhao

Domain Adaptation for Question Answering via Question Classification (2022.coling-1)

Challenge:	Existing QA datasets only contain unconditional and parallel answers . conditional question answering with hierarchical multi-span answers is challenging for the community to solve .
Approach:	They propose a conditional question answering task with hierarchical multi-span answers . they propose CMQA, which contains conditional and hierarchic samples .
Outcome:	The proposed task can be used to build more reliable and sophisticated QA systems.

Copied to clipboard

Zhenrui Yue, Huimin Zeng, Ziyi Kou, Lanyu Shang, Dong Wang

Leveraging fine-tuned Large Language Models with LoRA for Effective Claim, Claimer, and Claim Object Detection (2024.eacl-long)

Challenge:	Question answering systems often experience performance deterioration upon user-generated questions.
Approach:	They propose a question classification framework to help QA domains adapt to different domains.
Outcome:	The proposed framework improves on state-of-the-art datasets against multiple datasets.

Copied to clipboard

Sotiris Kotitsas, Panagiotis Kounoudis, Eleni Koutli, Haris Papageorgiou

Training Language Models to Generate Text with Citations via Fine-grained Rewards (2024.acl-long)

Challenge:	Existing work on identifying claims has focused on sentence level, neglecting supplementary attributes such as the claimer and claim object of the claim.
Approach:	They propose a novel approach to detect claims using large language models in natural language understanding and text generation.
Outcome:	The proposed approach transforms claim, claimer and claim object detection task into QA setting.

Copied to clipboard

Chengyu Huang, Zeqiu Wu, Yushi Hu, Wenya Wang

Benchmarking Hallucination in Large Language Models Based on Unanswerable Math Word Problem (2024.lrec-main)

Challenge:	Recent Large Language Models (LLMs) are prone to hallucination and their outputs often contain incorrect or unverifiable claims.
Approach:	They propose a training framework using fine-grained rewards to teach LLMs to generate highly supportive and relevant citations while ensuring the correctness of their responses.
Outcome:	The proposed training framework outperforms existing methods on QA datasets and surpasses GPT-3.5-turbo on LLaMA-2-7B.

Copied to clipboard

YuHong Sun, Zhangyue Yin, Qipeng Guo, Jiawen Wu, Xipeng Qiu, Hui Zhao

The Problem of Ambiguity in Table Question Answering (2026.findings-eacl)

Challenge:	Large language models (LLMs) are highly effective in various natural language processing tasks, but can produce unreliable conjectures in ambiguous contexts, which is known as hallucination.
Approach:	They propose a method to evaluate LLM hallucination in Question Answering based on the unanswerable math word problem (UMWP) . they combine text similarity and mathematical expression detection to determine whether LLM considers the question unanswered.
Outcome:	The proposed method combines text similarity and mathematical expression detection to determine whether the LLM considers the question unanswerable.

Copied to clipboard

Jorge Osés Grijalba, L. Alfonso Ureña, Eugenio Martínez-Cámara, Jose Camacho-Collados

Language Models Still Struggle to Zero-shot Reason about Time Series (2024.findings-emnlp)

Challenge:	Existing approaches to question answering on tabular data have limited capabilities due to ambiguousness inherent to tabular datasets.
Approach:	They propose to use large language models to answer questions on tabular data by analyzing tabular tables and detecting ambiguity.
Outcome:	The proposed model can detect ambiguity in tabular data and provide an initial ground for a deeper discussion on how to approach it in the age of LLMs.

Copied to clipboard

Mike Merrill, Mingtian Tan, Vinayak Gupta, Thomas Hartvigsen, Tim Althoff

Weakly-Supervised Questions for Zero-Shot Relation Extraction (2023.eacl-main)

Challenge:	Time series are critical for decision-making in fields like finance and healthcare.
Approach:	They propose a framework for time series reasoning that includes formal tasks and a dataset of multi-scale time series paired with text captions across ten domains.
Outcome:	The proposed framework combines formal tasks and a dataset of multi-scale time series paired with text captions across ten domains to examine whether language models achieve three forms of reasoning.

Copied to clipboard

Saeed Najafi, Alona Fyshe

Modelling Long-distance Node Relations for KBQA with Global Dynamic Graph (2020.coling-main)

Challenge:	Zero-Shot Relation Extraction (ZRE) is a task where the training and test sets have no shared relation types.
Approach:	They propose to learn a model that can translate relation descriptions into relevant questions, which are then leveraged to generate the correct tail entity.
Outcome:	The proposed model outperforms the state-of-the-art on the fewrel and WikiZSL datasets by more than 16 F1 points without using gold question templates.

Copied to clipboard

Xu Wang, Shuai Zhao, Jiale Han, Bo Cheng, Hao Yang, Jianchang Ao, Zhenzi Li

Stress Test Evaluation of Transformer-based Models in Natural Language Understanding Tasks (2020.lrec-1)

Challenge:	Existing studies rely on deep graph neural networks (GNNs) to capture rich structural information, but they lack the structural information needed for QA.
Approach:	They propose a framework which captures structural information from KBs and models long-distance node relations from two perspectives.
Outcome:	The proposed framework models long-distance node relations from two perspectives . it is based on two widely used multi-hop KBQA datasets .

Copied to clipboard

Carlos Aspillaga, Andrés Carvallo, Vladimir Araujo

FQuAD2.0: French Question Answering and Learning When You Don’t Know (2022.lrec-1)

Challenge:	Existing models are weak and take advantage of failures and errors in datasets to improve performance.
Approach:	They evaluate three Transformer-based models in Natural Language Inference and Question Answering tasks to see if they are more robust or have the same flaws as their predecessors.
Outcome:	The proposed models outperform recurrent neural network models to stress tests on both NLI and QA tasks.

Copied to clipboard

Quentin Heinrich, Gautier Viaud, Wacim Belblidia

CAM 2.0: End-to-End Open Domain Comparative Question Answering System (2024.lrec-main)

Challenge:	Question Answering, including Reading Comprehension, has seen significant scientific breakthroughs over the past few years . but most of these breakthroughs are centered on the English language .
Approach:	They propose a dataset to train Question Answering models in the French language . they extend the dataset to 17,000+ unanswerable questions annotated adversarially .
Outcome:	The proposed dataset makes it possible to train French Question Answering models with the ability to distinguish unanswerable questions from answerable ones.

Copied to clipboard

Ahmad Shallouf, Hanna Herasimchyk, Mikhail Salnikov, Rudy Alexandro Garrido Veliz, Natia Mestvirishvili, Alexander Panchenko, Chris Biemann, Irina Nikishina

QuAC: Question Answering in Context (D18-1)

Challenge:	Comparative Question Answering is a Natural Language Processing task that combines Question Answers and Argument Mining.
Approach:	They propose a system for answering comparative questions called CAM 2.0 and a public leaderboard called CompUGE that unifies existing datasets under a single easy-to-use evaluation suite.
Outcome:	The proposed system is compared with previous web-form-based systems . it features question identification, object and aspect labeling, stance classification, summarization . the proposed system has a user-friendly interface and is available for free on the web .

Copied to clipboard

Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, Luke Zettlemoyer

ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answers (2022.acl-long)

Challenge:	a dataset for Question Answering in Context contains 14K information-seeking QA dialogs . questions are often more open-ended, unanswerable, or only meaningful within the dialog context .
Approach:	They propose a dataset for Question Answering in Context that contains 14K dialogs . they use a student to ask questions about a Wikipedia section and a teacher to answer them .
Outcome:	The proposed dataset underperforms humans in a number of reference models . the dataset contains 14K information-seeking dialogs over sections from Wikipedia .

Copied to clipboard

Haitian Sun, William Cohen, Ruslan Salakhutdinov

TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance (2021.acl-long)

Challenge:	Existing datasets for reading comprehension have deterministic answers, but questions in the real world do not always have definite answers.
Approach:	They propose a Question Answering (QA) dataset that contains complex questions with conditional answers.
Outcome:	The proposed dataset will motivate further research in answering complex questions over long documents.

Copied to clipboard

Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, Tat-Seng Chua

Interactive Language Learning by Question Answering (D19-1)

Challenge:	Existing QA systems focus on unstructured text, structured knowledge base, or semi-structured tables.
Approach:	They propose a large-scale question answering model based on financial reports . numerical reasoning is usually required to infer the answer .
Outcome:	The proposed model achieves 58.0% inF1, an 11.1% increase over the baseline model, but still lags behind the best human model.

Copied to clipboard

Xingdi Yuan, Marc-Alexandre Côté, Jie Fu, Zhouhan Lin, Chris Pal, Yoshua Bengio, Adam Trischler

RoD-TAL: A Benchmark for Answering Questions in Romanian Driving License Exams (2026.findings-eacl)

Challenge:	Existing machine reading comprehension tasks lack interactive information-seeking component of comprehension.
Approach:	They propose a question-asking task that asks questions in a text-based environment . they propose QAit, which uses a game generator to build models that include deep reinforcement learning agents.
Outcome:	The proposed task poses questions about existence, location, and attributes of objects found in environment.

Copied to clipboard

Andrei Vlad Man, Răzvan-Alexandru Smădu, Cristian-George Craciun, Dumitru-Clementin Cercel, Florin Pop, Mihaela-Claudia Cercel

FiD-Ex: Improving Sequence-to-Sequence Models for Extractive Rationale Generation (2021.emnlp-main)

Challenge:	a growing need for tools that support legal education, especially in under-resourced languages such as Romanian . we evaluate the capabilities of large language models and vision-language models in legal education .
Approach:	They evaluate the capabilities of Large Language Models and Vision-Language Models in Romanian driving law through textual and visual question-answering tasks.
Outcome:	The proposed model improves retrieval performance and QA accuracy in Romanian driving tests.

Copied to clipboard

Kushal Lakhotia, Bhargavi Paranjape, Asish Ghoshal, Scott Yih, Yashar Mehdad, Srini Iyer

Repurposing Entailment for Multi-Hop Question Answering Tasks (N19-1)

Challenge:	Pre-trained sequence to sequence models are effective in making and generating NL explanations, but they have many shortcomings.
Approach:	They propose a model that uses sentence markers to eliminate explanation fabrication . they use fusion-in-decoder architecture to handle long input contexts .
Outcome:	The proposed model significantly improves on the ERASER explainability benchmark.

Copied to clipboard

Harsh Trivedi, Heeyoung Kwon, Tushar Khot, Ashish Sabharwal, Niranjan Balasubramanian

Handling Anomalies of Synthetic Questions in Unsupervised Question Answering (2020.coling-main)

Challenge:	Existing approaches to use entailment models for question answering are limited . large scale datasets are typically framed at a sentence level, whereas question answering requires verifying whether multiple sentences, taken together as a premise, entitle a hypothesis.
Approach:	They propose a general architecture that can use entailment models for multi-hop QA tasks.
Outcome:	The proposed model outperforms QA models trained on target datasets and the OpenAI transformer models.

Copied to clipboard

Giwon Hong, Junmo Kang, Doyeon Lim, Sung-Hyon Myaeng

ProQA: Structural Prompt-based Pre-training for Unified Question Answering (2022.naacl-main)

Challenge:	Existing approaches to improve unsupervised Question Answering (UQA) are expensive and require additional datasets.
Approach:	They propose an unsupervised QA approach that generates QA training data automatically.
Outcome:	The proposed method improves unsupervised QA significantly across a number of QA tasks.

Copied to clipboard

Wanjun Zhong, Yifan Gao, Ning Ding, Yujia Qin, Zhiyuan Liu, Ming Zhou, Jiahai Wang, Jian Yin, Nan Duan

Adaptive Retrieval Without Self-Knowledge? Bringing Uncertainty Back Home (2025.acl-long)

Challenge:	Existing QA research on question answering is focused on specific question types, knowledge domains, or reasoning skills.
Approach:	They propose a unified QA paradigm that solves various tasks through a single model.
Outcome:	The proposed model improves QA-centric ability on 11 QA benchmarks.

Copied to clipboard

Viktor Moskvoretskii, Maria Marina, Mikhail Salnikov, Nikolay Ivanov, Sergey Pletenev, Daria Galimzianova, Nikita Krayko, Vasily Konovalov, Irina Nikishina, Alexander Panchenko

Answers Unite! Unsupervised Metrics for Reinforced Summarization Models (D19-1)

Challenge:	Recent adaptive retrieval methods integrate LLMs’ intrinsic knowledge with external information appealing to LLM self-knowledge, but they often neglect efficiency evaluations and comparisons with uncertainty estimation techniques.
Approach:	They propose to integrate LLMs’ intrinsic knowledge with external information appealing to LLM self-knowledge but neglect efficiency evaluations and comparisons with uncertainty estimation techniques.
Outcome:	The proposed methods outperform complex pipelines in terms of efficiency and self-knowledge while maintaining comparable QA performance.

Copied to clipboard

Thomas Scialom, Sylvain Lamprier, Benjamin Piwowarski, Jacopo Staiano

GLUECoS: An Evaluation Benchmark for Code-Switched NLP (2020.acl-main)

Challenge:	Abstractive summarization approaches based on Reinforcement Learning (RL) have been proposed to overcome classical likelihood maximization.
Approach:	They propose to use Reinforcement Learning to learn the model parameters through RL techniques to overcome classical likelihood maximization.
Outcome:	The proposed measures favor ROUGE with the additional property of not requiring reference summaries.

Copied to clipboard

Simran Khanuja, Sandipan Dandapat, Anirudh Srinivasan, Sunayana Sitaram, Monojit Choudhury

Multilinguality

Challenge:	Recent studies show multilingual contextual embedding models perform better on cross-lingual and multilingual tasks.
Approach:	They propose to evaluate multilingual contextual embedding models on multilingual data . they use language identification from text, POS tagging, Named Entity Recognition and Question Answering .
Outcome:	The proposed benchmark evaluates models on language identification from text, POS tagging, Named Entity Recognition, Question Answering and a new task for code-switching, Natural Language Inference.

Beyond Timestamps: Bridging Forward and Backward Reasoning in Temporal Numerical and Relational Understanding (2026.acl-long)

Copied to clipboard

Xinying Qian, Ying Zhang, Xuhui Sui, Yu Zhao, Baohang Zhou, Jeff Z. Pan

Improving Unsupervised Question Answering via Summarization-Informed Question Generation (2021.emnlp-main)

Challenge:	Existing benchmarks for Temporal Numerical and Relational reasoning rely on single-task evaluation paradigms.
Approach:	They propose a benchmark to evaluate Temporal Numerical and Relational reasoning . they propose QA and verification, and a Consistency Rate to quantify robustness .
Outcome:	The proposed framework evaluates both Temporal Numerical and Relational reasoning . it measures the alignment between QA and FV and the Consistency Rate measures robustness across these directions.

Copied to clipboard

Chenyang Lyu, Lifeng Shang, Yvette Graham, Jennifer Foster, Xin Jiang, Qun Liu

CoCoA: Confidence- and Context-Aware Adaptive Decoding for Resolving Knowledge Conflicts in Large Language Models (2025.emnlp-main)

Challenge:	Question Generation (QG) is the production of meaningful questions given a set of input passages and corresponding answers.
Approach:	They propose a method which uses questions generated heuristically from news summaries as a source of training data for a QG system.
Outcome:	The proposed method outperforms previous unsupervised models on three in-domain datasets and three out-of-domain ones.

Copied to clipboard

Anant Khandelwal, Manish Gupta, Puneet Agrawal

Continuous Language Generative Flow (2021.acl-long)

Challenge:	Existing contrastive decoding methods that handle conflict lack adaptability and can degrade performance in low conflict settings.
Approach:	They propose a token-level algorithm for principled conflict resolution and enhanced faithfulness that resolves conflict by utilizing confidence-aware measures and the generalized divergence between parametric and contextual distributions.
Outcome:	The proposed algorithm achieves 9.2 points on average in QA, summarization, and long-form question answering (LFQA) benchmarks and improves factuality by 2.5 points on the key benchmarks.

Copied to clipboard

Zineng Tang, Shiyue Zhang, Hyounghun Kim, Mohit Bansal

INVITE: a Testbed of Automatically Generated Invalid Questions to Evaluate Large Language Models for Hallucinations (2023.findings-emnlp)

Challenge:	Recent years have witnessed various types of generative models for natural language generation (NLG), especially RNNs or transformers.
Approach:	They propose a flow-based language generation model that adapts flow-derived generative models to language generation via continuous input embeddings, adapted affine coupling structures, and a novel architecture for autoregressive text generation.
Outcome:	The proposed model improves on QG and NMT and improves performance over baselines on SQuAD and TVQA and NML16.

Copied to clipboard

Anil Ramakrishna, Rahul Gupta, Jens Lehmann, Morteza Ziyadi

GraphOTTER: Evolving LLM-based Graph Reasoning for Complex Table Question Answering (2025.coling-main)

Challenge:	Recent advances in Large language models have enabled them to hold free form conversations over multiple turns, but they exhibit a tendency to make unfounded and incorrect statements, commonly labeled as hallucinations.
Approach:	They propose a framework to test large language models for hallucinations using automatically generated INValId questions.
Outcome:	The proposed framework is based on a testbed of automatically generated INValId questions to evaluate large language models for hallucinations.

Copied to clipboard

Qianlong Li, Chen Huang, Shuai Li, Yuanxin Xiang, Deng Xiong, Wenqiang Lei

Answer Generation for Retrieval-based Question Answering Systems (2021.findings-acl)

Challenge:	Existing methods for complex table question answering are often implicit, feeding the entire table into prompts.
Approach:	They propose a GraphOTTER that explicitly establishes the reasoning process to pinpoint the correct answers.
Outcome:	The proposed method is able to identify the correct answers on two benchmark datasets and two LLM backbones.

Copied to clipboard

Chao-Chun Hsu, Eric Lind, Luca Soldaini, Alessandro Moschitti

On Knowledge distillation from complex networks for response prediction (N19-1)

Challenge:	Question Answering systems are a core component of many commercial applications . answer sentence selection (AS2) models are trained to select the best answer sentence .
Approach:	They propose to train a sequence to sequence transformer model to generate an answer from a set of candidates.
Outcome:	The proposed model improves accuracy by 32 points over the state-of-the-art model on English AS2 datasets.

Copied to clipboard

Siddhartha Arora, Mitesh M. Khapra, Harish G. Ramaswamy

QACE: Asking Questions to Evaluate an Image Caption (2021.findings-emnlp)

Challenge:	Recent advances in Question Answering have led to the development of very complex models . however, these models are expensive in space and time and require limited resources .
Approach:	They propose to use simple models which learn to emulate characteristics of a teacher network . they use a 12GB Tesla K80 GPU to restrict the maximum length of the input document .
Outcome:	The proposed model can perform better on a Holl-E dialog dataset.

Copied to clipboard

Hwanhee Lee, Thomas Scialom, Seunghyun Yoon, Franck Dernoncourt, Kyomin Jung

GBT: Generative Boosting Training Approach for Paraphrase Identification (2023.findings-emnlp)

Challenge:	Existing metric for image captioning evaluation is based on n-gram similarity metrics but these fail to capture semantic errors in captions.
Approach:	They propose a new metric based on Question Answering for Caption Evaluation to evaluate image captioning based upon Question Generation and Question Answers systems.
Outcome:	The proposed metric is multi-modal, reference-less and explainable.

Copied to clipboard

Rui Peng, Zhiling Jin, Yu Hong

LINKAGE: Listwise Ranking among Varied-Quality References for Non-Factoid QA Evaluation via LLMs (2024.findings-emnlp)

Challenge:	Paraphrase Identification (PI) is a fundamental natural language understanding task with non-trivial challenges.
Approach:	They propose a Generative Boosting Training approach for Paraphrase Identification (PI) they use a seq2seq model to perform DA on misclassified instances periodically .
Outcome:	The proposed method outperforms state-of-the-art PI models on English and Chinese PI tasks with good efficiency and effectiveness.

Copied to clipboard

Sihui Yang, Keping Bi, Wanqing Cui, Jiafeng Guo, Xueqi Cheng

AVA: an Automatic eValuation Approach for Question Answering Systems (2021.naacl-main)

Challenge:	Non-factoid (NF) question answering is challenging to evaluate due to diverse potential answers and no objective criterion.
Approach:	They propose a listwise NFQA evaluation approach that uses Large Language Models to rank candidate answers in a descending list of reference answers sorted by descending quality.
Outcome:	The proposed method has higher correlations with human annotations than standard methods.

Copied to clipboard

Thuy Vu, Alessandro Moschitti

Improving Multi-hop Question Answering over Knowledge Graphs using Knowledge Base Embeddings (2020.acl-main)

Challenge:	AVA is an automatic evaluation approach for question answering . it uses transformer-based language models to encode question, answer, and reference texts .
Approach:	They propose an automatic evaluation approach for Question Answering that uses Transformer-based language models to encode question, answer, and reference texts.
Outcome:	AVA can estimate system Accuracy with an error lower than 7% at 95% confidence level . the proposed approach achieves 74.7% F1 score in predicting human judgment for single answers .

Copied to clipboard

Apoorv Saxena, Aditay Tripathi, Partha Talukdar

Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering (2020.acl-main)

Challenge:	Existing multi-hop KGQA methods impose heuristic neighborhood limits, which often make it much harder to answer the input NL question.
Approach:	They propose to use knowledge Graphs (KG) to answer natural language queries over the KG.
Outcome:	The proposed method is particularly effective in performing multi-hop KGQA over sparse KGs.

Copied to clipboard

Alexander Fabbri, Patrick Ng, Zhiguo Wang, Ramesh Nallapati, Bing Xiang

Towards Adaptive Memory-Based Optimization for Enhanced Retrieval-Augmented Generation (2025.findings-acl)

Challenge:	Question Answering (QA) is a field of increasing demand due to the availability of information online.
Approach:	They propose an unsupervised approach to training QA models with generated pseudo-training data by applying a simple template on a related sentence rather than the original context sentence.
Outcome:	The proposed approach improves the performance of a QA model on generated pseudo-training data.

Copied to clipboard

Qitao Qin, Yucong Luo, Yihang Lu, Zhibo Chu, Xiaoman Liu, Xianwei Meng

SynDARin: Synthesising Datasets for Automated Reasoning in Low-Resource Languages (2025.coling-main)

Challenge:	Existing methods for enhancing response accuracy and accuracy struggle with open-domain QA tasks because they perform independent retrieval operations without maintaining a summarizing memory or using adaptive retrieval strategies.
Approach:	They propose a method that integrates non-parametric knowledge from external knowledge bases into models to enhance response accuracy while mitigating factual errors and hallucinations.
Outcome:	The proposed method improves on open-domain QA datasets and reduces noise and hallucinations due to redundant information and insufficient information integration.

Copied to clipboard

Gayane Ghazaryan, Erik Arakelyan, Isabelle Augenstein, Pasquale Minervini

Machine Translation

Challenge:	Question Answering datasets are scarce for languages other than English due to the cost and difficulties of collection and manual annotation.
Approach:	They propose a method for generating and validating QA datasets for low-resource languages . they use English data as context to generate synthetic multiple-choice (MC) question-answer pairs .
Outcome:	The proposed method maintains quality, reduces likelihood of factual errors, and circumvents costly annotation.

Incorporating medical knowledge in BERT for clinical relation extraction (2021.emnlp-main)

Copied to clipboard

Arpita Roy, Shimei Pan

Are Multilingual Models the Best Choice for Moderately Under-resourced Languages? A Comprehensive Assessment for Catalan (2021.findings-acl)

Challenge:	Pre-trained language models (PLMs) are used for diverse NLP tasks such as Information Extraction, Sentiment Analysis and Question/Answering.
Approach:	They propose to add medical knowledge to pre-trained language models to facilitate clinical relation extraction using a large text corpus.
Outcome:	The proposed model outperforms the state-of-the-art systems on the benchmark i2b2/VA 2010 clinical relation extraction dataset.

Copied to clipboard

Jordi Armengol-Estapé, Casimiro Pio Carrino, Carlos Rodriguez-Penagos, Ona de Gibert Bonet, Carme Armentano-Oller, Aitor Gonzalez-Agirre, Maite Melero, Marta Villegas

ISAAQ - Mastering Textbook Questions with Pre-trained Transformers and Bottom-Up and Top-Down Attention (2020.emnlp-main)

Challenge:	Multilingual language models have been a crucial breakthrough for under-resourced languages . however, the superiority of language-specific models has already been proven for underresourced ones .
Approach:	They propose to build a monolingual monolingual model that is comparable to state-of-the-art large multilingual models.
Outcome:	The proposed model consistently outperforms state-of-the-art models across tasks and settings.

Copied to clipboard

Jose Manuel Gomez-Perez, Raúl Ortega

Coreference Reasoning in Machine Reading Comprehension (2021.acl-long)

Challenge:	Textbook Question Answering is a complex task that requires reasoning with multimodal information from text and diagrams.
Approach:	They propose to use transformer language models and bottom-up and top-down attention to tackle the language and visual understanding challenges of text and diagrams.
Outcome:	The proposed system achieves unprecedented accuracies on all TQA question types . the system also obtains state-of-the-art results in other demanding datasets .

Copied to clipboard

Mingzhu Wu, Nafise Sadat Moosavi, Dan Roth, Iryna Gurevych

DHP Benchmark: Are LLMs Good NLG Evaluators? (2025.findings-naacl)

Challenge:	Existing datasets for machine reading comprehension do not reflect the natural distribution and, consequently, the challenges of coreference reasoning.
Approach:	They propose to use existing coreference resolution datasets to train machine reading comprehension models to better reflect the natural distribution and, consequently, the challenges of coreference reasoning.
Outcome:	The proposed method improves the performance of state-of-the-art models on a set of coreference-related datasets.

Copied to clipboard

Yicheng Wang, Jiayi Yuan, Yu-Neng Chuang, Zhuoer Wang, Yingchi Liu, Mark Cusick, Param Kulkarni, Zhengping Ji, Yasser Ibrahim, Xia Hu

Joint Multitask Learning for Community Question Answering Using Task-Specific Embeddings (D18-1)

Challenge:	Large Language Models (LLMs) are increasingly serving as evaluators in Natural Language Generation (NLG) tasks.
Approach:	They propose a framework that measures the discernment of Large Language Models (LLMs) across diverse NLG tasks.
Outcome:	The proposed framework provides quantitative discernment scores for LLMs across four NLG tasks.

Copied to clipboard

Shafiq Joty, Lluís Màrquez, Preslav Nakov

Representation Learning

Challenge:	Stack-Overflow, Quora, and Yahoo! Answers forums are not moderated, which results in noisy and redundant content.
Approach:	They use deep neural networks to learn meaningful task-specific embeddings . they incorporate the embeddables into a conditional random field model .
Outcome:	The proposed task improves significantly across evaluation metrics.

A Study on Efficiency, Accuracy and Document Structure for Answer Sentence Selection (2020.coling-main)

Copied to clipboard

Daniele Bonadiman, Alessandro Moschitti

CNER: Concept and Named Entity Recognition (2024.naacl-long)

Challenge:	Existing approaches to QA re-rank sentences use huge neural models or complex attentive architectures.
Approach:	They propose to exploit the intrinsic structure of the original rank with an effective word-relatedness encoder to achieve the highest accuracy among the cost-efficient models.
Outcome:	The proposed model takes 9.5 seconds to train on the WikiQA dataset, compared with 18 minutes required by a standard BERT-base fine-tuning.

Copied to clipboard

Giuliano Martinelli, Francesco Molfese, Simone Tedeschi, Alberte Fernández-Castro, Roberto Navigli

Guiding the Growth: Difficulty-Controllable Question Generation through Step-by-Step Rewriting (2021.acl-long)

Challenge:	Concept and Named Entity Recognition (CNER) is a new unified task that handles concepts and entities mentioned in unstructured texts seamlessly.
Approach:	They propose a new unified task that handles concepts and entities mentioned in unstructured texts seamlessly.
Outcome:	The proposed task gains +5.4 and +8 macro F1 points when performed as a unified task compared to specialized named entity and concept recognition systems.

Copied to clipboard

Yi Cheng, Siyao Li, Bang Liu, Ruihui Zhao, Sujian Li, Chenghua Lin, Yefeng Zheng

(QA)2: Question Answering with Questionable Assumptions (2023.acl-long)

Challenge:	Existing QG systems perform substantially worse in answering multi-hop questions than single-hop ones.
Approach:	They propose a framework that progressively increases question difficulty through step-by-step rewriting under the guidance of an extracted reasoning chain.
Outcome:	The proposed framework increases question difficulty through step-by-step rewriting under the guidance of an extracted reasoning chain.

Copied to clipboard

Najoung Kim, Phu Mon Htut, Samuel R. Bowman, Jackson Petty

S2LPP: Small-to-Large Prompt Prediction across LLMs (2025.findings-emnlp)

Challenge:	a number of questions contain questionable assumptions, such as when did Marie Curie discover Uranium, that cannot be answered as a true when question.
Approach:	They propose an open-domain evaluation dataset that can detect questionable assumptions . they propose a method that can be used to produce adequate responses for questions with questionable assumption.
Outcome:	The proposed model detects questionable assumptions and produces adequate responses for both types of questions.

Copied to clipboard

Liang Cheng, Tianyi Li, Zhaowei Wang, Mark Steedman

Unsupervised Question Answering by Cloze Translation (P19-1)

Challenge:	a small model can be used to select effective prompt templates for a larger model.
Approach:	They propose a method to use a smaller model to select effective prompt templates for a larger model.
Outcome:	The proposed method significantly reduces the cost of prompt engineering while matching performance with optimal prompts among candidates.

Copied to clipboard

Patrick Lewis, Ludovic Denoyer, Sebastian Riedel

Mitigating False-Negative Contexts in Multi-document Question Answering with Retrieval Marginalization (2021.emnlp-main)

Challenge:	Existing QA datasets only available for limited domains and languages.
Approach:	They propose to generate context, question and answer triples in an unsupervised manner and synthesize extractive QA training data automatically.
Outcome:	The proposed approach outperforms existing QA models on a common EQA benchmark dataset.

Copied to clipboard

Ansong Ni, Matt Gardner, Pradeep Dasigi

Building a Dataset for Automatically Learning to Detect Questions Requiring Clarification (2022.lrec-1)

Challenge:	Question Answering models typically use retrieval and reasoning components to identify relevant information for reasoning.
Approach:	They propose a retrieval parameterization method that marginalizes unanswerable queries . they show that marginalization allows a model to mitigate false negatives in annotations .
Outcome:	The proposed model improves on two multi-document question answering datasets and shows that marginalization improves performance.

Copied to clipboard

Ivano Lauriola, Kevin Small, Alessandro Moschitti

Question Answering Over Temporal Knowledge Graphs (2021.acl-long)

Challenge:	Existing work on question answering systems assumes all questions are intelligible and unambiguous . however, available datasets do not meet requirements for building commercial virtual assistants .
Approach:	They propose to make question answering systems more robust by classifying if question is intelligible and returning a clarification question for contextual ambiguity.
Outcome:	The proposed system can classify if the input question is intelligible and return a clarification question for ambiguous questions.

Copied to clipboard

Apoorv Saxena, Soumen Chakrabarti, Partha Talukdar

T-REx: A Large Scale Alignment of Natural Language with Knowledge Base Triples (L18-1)

Challenge:	Temporal Knowledge Graphs (Temporal KGs) provide temporal scopes (start and end times) on each edge in the Knowledge . Lack of broad coverage datasets has been limiting progress in this area .
Approach:	They propose a transformer-based solution that exploits recent advances in Temporal Knowledge Graph embeddings and achieves an increase of 120% in accuracy over the next best performing method.
Outcome:	The proposed solution improves on the only known dataset by 340x . it increases accuracy by 120% over the baseline solution .

Copied to clipboard

Hady Elsahar, Pavlos Vougiouklis, Arslen Remaci, Christophe Gravier, Jonathon Hare, Frederique Laforest, Elena Simperl

Knowledge Representation

Challenge:	Existing datasets that provide alignments between natural language and knowledge bases (KB) triples are limited in size, lack coverage and are of unreported quality.
Approach:	They propose to build a large scale dataset of alignments between Wikipedia abstracts and Wikidata triples that is two orders of magnitude larger than the largest available alignments dataset.
Outcome:	The proposed dataset is two orders of magnitude larger than the largest available dataset and covers 2.5 times more predicates.

Table Question Answering in the Era of Large Language Models: A Comprehensive Survey of Tasks, Methods, and Evaluation (2026.acl-long)

Copied to clipboard

Wei Zhou, Bolei Ma, Annemarie Friedrich, Mohsen Mesgar

Will this Question be Answered? Question Filtering via Answer Model Distillation for Efficient Question Answering (2021.emnlp-main)

Challenge:	Table Question Answering (TQA) aims to answer natural language questions using tabular data.
Approach:	They propose a systematic overview of TQA research using large language models and summarize available benchmarks based on task features.
Outcome:	The proposed framework provides a comprehensive overview of the current state of the art in the field of Table Question Answering.

Copied to clipboard

Siddhant Garg, Alessandro Moschitti

Rethinking Table Pruning in TableQA: From Sequential Revisions to Gold Trajectory-Supervised Parallel Search (2026.acl-long)

Challenge:	Existing methods to improve QA efficiency do not take specific answers into account.
Approach:	They propose a transformer-based approach to improve QA efficiency by filtering out questions that will not be answered by the system.
Outcome:	The proposed model can approximate the Precision/Recall curves of the target QA system.

Copied to clipboard

Yu Guo, Shenghao Ye, Shuangwu Chen, Zijian Wen, Tao Zhang, Bai Qirui, Dong Jin, Yunpeng Hou, Huasen He, null Jianyang, Xiaobin Tan

Harvesting and Refining Question-Answer Pairs for Unsupervised QA (2020.acl-main)

Challenge:	Existing pruning methods rely on sequential revisions and unreliable critique signals . Existing methods fail to detect the loss of answer-critical data .
Approach:	They propose a table pruning framework which transforms table pruning to gold trajectory-supervised parallel search.
Outcome:	The proposed framework outperforms the strongest baseline pruning framework by 3.2% on various tabular reasoning tasks.

Copied to clipboard

Zhongli Li, Wenhui Wang, Li Dong, Furu Wei, Ke Xu

ReasTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examples (2022.emnlp-main)

Challenge:	Recent research attempts to extend unsupervised question answering to settings with few or no labeled data available.
Approach:	They propose two approaches to improve unsupervised question answering . first, they harvest lexically and syntactically divergent Wikipedia questions to automatically construct a corpus of question-answer pairs . second, they take advantage of the QA model to extract more appropriate answers .
Outcome:	The proposed approach outperforms previous unsupervised approaches by a large margin and is competitive with early supervised models.

Copied to clipboard

Yilun Zhao, Linyong Nan, Zhenting Qi, Rui Zhang, Dragomir Radev

FinChart-Bench: Benchmarking Financial Chart Comprehension in Vision-Language Models (2026.acl-long)

Challenge:	Existing models with table-specific architectures and pre-training methods perform well on understanding table structures but lack table reasoning skills.
Approach:	They propose to pre-train tables with table reasoning skills without complex architectures . they define 7 table reasoning skill, and then pre-teach them to generate tables .
Outcome:	The proposed model improves on four tasks and is available on github.

Copied to clipboard

Dong Shu, Haoyang Yuan, Yuchen Wang, Yanguang Liu, Huopu Zhang, Mengnan Du

Knowledge Transfer from Answer Ranking to Answer Generation (2022.emnlp-main)

Challenge:	FinChart-Bench is the first benchmark specifically focused on real-world financial charts.
Approach:	They propose a benchmark specifically focused on real-world financial charts.
Outcome:	The proposed benchmark evaluates 26 state-of-the-art LVLMs on FinChart-Bench.

Copied to clipboard

Matteo Gabburo, Rik Koncel-Kedziorski, Siddhant Garg, Luca Soldaini, Alessandro Moschitti

Project PIAF: Building a Native French Question-Answering Dataset (2020.lrec-1)

Challenge:	Recent studies show that Question Answering (QA) based on Answer Sentence Selection (AS2) can be improved by generating an improved answer from the top-k ranked answer sentences.
Approach:	They propose to train a GenQA model by transferring knowledge from a trained AS2 model . they use top ranked candidate as the generation target and next k top rated candidates as context .
Outcome:	The proposed model outperforms existing models on public and industrial datasets.

Copied to clipboard

Rachel Keraron, Guillaume Lancrenon, Mathilde Bras, Frédéric Allary, Gilles Moyse, Thomas Scialom, Edmundo-Pavel Soriano-Morales, Jacopo Staiano

mRAKL: Multilingual Retrieval-Augmented Knowledge Graph Construction for Low-Resourced Languages (2025.findings-acl)

Challenge:	a lack of data for non-English languages is limiting the development of downstream tasks such as Question Answering.
Approach:	They propose to collect a native French Question Answering Dataset using a participatory setup.
Outcome:	The proposed tool allows volunteers to participate in crowdsourced annotations in French.

Copied to clipboard

Hellina Hailu Nigatu, Min Li, Maartje Ter Hoeve, Saloni Potdar, Sarah Chasins

Knowledge Representation

Challenge:	Knowledge Graphs are structured multirelational graphs that store factual knowledge.
Approach:	They introduce a Retrieval-Augmented Generation (mRAKL) based system to perform mKGC.
Outcome:	The proposed approach improves over a no-context setting with an idealized retrieval system.

MLEC-QA: A Chinese Multi-Choice Biomedical Question Answering Dataset (2021.emnlp-main)

Copied to clipboard

Jing Li, Shangping Zhong, Kaizhi Chen

CCQA: Generating Question from Solution Can Improve Inference-Time Reasoning in SLMs (2025.emnlp-main)

Challenge:	Question Answering (QA) is a branch of QA that enables effective perceiving, accessing, and understanding complex biomedical knowledge by innovative applications.
Approach:	They present MLEC-QA, the largest-scale Chinese multi-choice biomedical QA dataset . they implement eight representative control methods and open-domain QA methods as baselines .
Outcome:	The proposed dataset is the largest-scale Chinese multi-choice biomedical QA dataset . it covers the following biomedically-relevant sub-fields: Clinic, Stomatology, Public Health, Traditional Chinese Medicine, and Traditional Chinese medicine Combined with Western Medicine.

Copied to clipboard

Jinyoung Kim, Ji Won Yoon

Context Filtering with Reward Modeling in Question Answering (2025.coling-main)

Challenge:	Recent advances in large language models (LLMs) have yielded remarkable performance across a wide range of tasks, including machine translation, code generation, sentiment analysis, and reasoning.
Approach:	They propose a new reasoning method that generates a question from each reasoning path and answer, evaluates each by its similarity to the original question, and selects the candidate solution with the highest similarity score as the final answer.
Outcome:	The proposed method outperforms existing state-of-the-art methods on mathematical and commonsense reasoning benchmarks and establishes a new practical baseline for efficient reasoning in SLMs.

Copied to clipboard

Sangryul Kim, James Thorne

CoPA: Benchmarking Personalized Question Answering with Data-Informed Cognitive Factors (2026.findings-acl)

Challenge:	Question Answering (QA) tasks require a mix of relevant and irrelevant information in these contexts to perform well.
Approach:	They propose a context filtering approach that removes non-essential details, summarizing crucial content through Reward Modeling.
Outcome:	The proposed approach outperforms baseline models in 6.8-folds.

Copied to clipboard

Hang Su, Zequn Liu, Chen Hu, Xuesong Lu, Yingce Xia, Liu Zhen

Biomedical Question Answering via Multi-Level Summarization on a Local Knowledge Graph (2026.acl-long)

Challenge:	Existing LLMs rely on surface-level similarity or manual heuristics to evaluate personalization . Existing evaluation protocols for personalization are lacking sufficient data-driven validation.
Approach:	They propose a benchmark to assess personalization by mining CIPDs to quantify individual preferences.
Outcome:	The proposed benchmark provides a more comprehensive and discriminative standard than generic metrics.

Copied to clipboard

Lingxiao Guan, Yuanhao Huang, Jie Liu

Lightweight Query Checkpoint: Classifying Faulty User Queries to Mitigate Hallucinations in Large Language Model Question Answering (2025.findings-acl)

Challenge:	Existing work on how to effectively capture multi-document relationships remains an open question . Existing techniques to mitigate this problem include hierarchical summarization of semantically related chunks or integrating Knowledge Graphs (KGs).
Approach:	They propose a method which constructs a local knowledge graph from retrieved documents . they use propositional claims to construct a knowledge graph and contextualize a small language model .
Outcome:	The proposed method outperforms RAG on biomedical benchmarks and is generalizable and effective.

Copied to clipboard

Minjoo Son, Jonghak Jang, Misuk Kim

More Bang for your Context: Virtual Documents for Question Answering over Long Documents (2024.findings-emnlp)

Challenge:	Question Answering (QA) with large language models has shown impressive performance, yet hallucinations persist . incorporating LQC into QA pipelines reduces hallucinosities while preserving strong answer quality.
Approach:	They propose a small classification model that detects verification-required queries before the LLM generates a potentially faulty answer.
Outcome:	The proposed model reduces hallucinations while maintaining strong answer quality.

Copied to clipboard

Yosi Mass, Boaz Carmeli, Asaf Yehudai, Assaf Toledo, Nathaniel Mills

How to Compare Things Properly? A Study of Argument Relevance in Comparative Question Answering (2025.acl-long)

Challenge:	Large language models struggle to utilize long contexts efficiently, resulting in a question answering problem.
Approach:	They propose a method to generate a short document that contains the most relevant parts for a given context window.
Outcome:	The proposed method improves the QA task by providing a short and focused VDoc to the LLM while keeping the context window full.

Copied to clipboard

Irina Nikishina, Saba Anwar, Nikolay Dolgov, Maria Manina, Daria Ignatenko, Artem Shelmanov, Chris Biemann

Discourse Comprehension: A Question Answering Framework to Represent Sentence Connections (2022.emnlp-main)

Challenge:	Comparative Question Answering (CQA) is a task that involves processing information and diverse viewpoints.
Approach:	They construct a dataset of arguments annotated with their relevance and use it to answer comparative questions.
Outcome:	The proposed dataset contains arguments annotated with their relevance and enables precise traceability and faithfulness.

Copied to clipboard

Wei-Jen Ko, Cutter Dalton, Mark Simmons, Eliza Fisher, Greg Durrett, Junyi Jessy Li

PakBBQ: A Culturally Adapted Bias Benchmark for QA (2025.emnlp-main)

Challenge:	Existing systems for text comprehension are inadequate for more holistic comprehension of a discourse.
Approach:	They propose a new paradigm that captures both discourse and semantic links between sentences in the form of free-form, open-ended questions.
Outcome:	The proposed model captures discourse and semantic links between sentences in the form of free-form, open-ended questions.

Copied to clipboard

Abdullah Hashmat, Muhammad Arham Mirza, Agha Ali Raza

On Synthesizing Data for Context Attribution in Question Answering (2025.acl-long)

Challenge:	Large Language Models (LLMs) are widely adopted in language processing applications, but they often perpetuate harmful societal biases.
Approach:	They propose a culturally and regionally adapted extension of the original Bias Benchmark for Question Answering dataset to address this gap.
Outcome:	The proposed model gains 12% accuracy with disambiguation and stronger counter bias behaviors in Urdu than in English.

Copied to clipboard

Gorjan Radevski, Kiril Gashteovski, Shahbaz Syed, Christopher Malon, Sebastien Nicolas, Chia-Chien Hung, Timo Sztyler, Verena Heußer, Wiem Ben Rim, Masafumi Enomoto, Kunihiro Takeoka, Masafumi Oyamada, Goran Glavaš, Carolin Lawrence

Dunhuang-Bench: How Well Do MLLMs Understand Cultural Heritage? (2026.findings-acl)

Challenge:	Large Language Models (LLMs) have a tendency to hallucinate, resulting in false or misleading answers.
Approach:	They propose a novel generative strategy for synthesizing context attribution data.
Outcome:	The proposed approach is highly effective for fine-tuning small LMs for context attribution in different QA tasks and domains.

Copied to clipboard

Junyi Yuan, Jian Zhang, Tianxiu Yu, Yanlin Zhou, Xiaobo Jin, Qiufeng Wang, Fangyu Wu

Metaphor and Large Language Models: When Surface Features Matter More than Deep Understanding (2025.findings-acl)

Challenge:	Recent advances in Multimodal Large Language Models (MLLMs) have led to extensive evaluations on Chinese cultural benchmarks.
Approach:	They construct a large-scale benchmark comprising 486 images and 22,970 QA pairs to evaluate MLLMs' cultural understanding.
Outcome:	The proposed benchmark incorporates three task formats to evaluate MLLMs’ cultural understanding: Question Answering with Text Description, Multi-turn Dialogue, and Question Answers with Choices.

Copied to clipboard

Elisa Sanchez-Bayona, Rodrigo Agerri

DBQR-QA: A Question Answering Dataset on a Hybrid of Database Querying and Reasoning (2024.findings-acl)

Challenge:	Existing studies on metaphor processing have focused on single datasets and specific task settings, often using artificially constructed data through lexical replacement.
Approach:	They propose to evaluate the capabilities of Large Language Models (LLMs) in metaphor interpretation across multiple datasets, tasks, and prompt configurations.
Outcome:	The proposed frameworks are more realistic and efficient than current models and are more efficient than existing models.

Copied to clipboard

Rungsiman Nararatwong, Chung-Chi Chen, Natthawut Kertkeidkachorn, Hiroya Takamura, Ryutaro Ichise

Large Language Models are Complex Table Parsers (2023.emnlp-main)

Challenge:	Question answering (QA) is a fundamental task in the field of Natural Language Processing (NLP).
Approach:	They propose a database querying and reasoning dataset for question answering that is designed to accommodate sequential questions and multi-hop queries.
Outcome:	The proposed dataset better mirrors the dynamics of real-world information retrieval and analysis with a particular focus on the financial reports of US companies.

Copied to clipboard

Bowen Zhao, Changkai Ji, Yuejie Zhang, Wen He, Yingwen Wang, Qing Wang, Rui Feng, Xiaobo Zhang

Multilingual Generation and Answering of Questions from Texts and Knowledge Graphs (2023.findings-emnlp)

Challenge:	Extensive experiments and results on Complex Table QA datasets, i.e., the open-domain dataset HiTAB and the aviation domain dataset AIT-QA show that our approach significantly outperforms previous work on both datasets.
Approach:	They propose to incorporate Generative Pre-trained Transformer 3.5 to address the specific challenges posed by Complex Table QA by reconstructing tables into tuples and using prompt templates to create dialogues.
Outcome:	The proposed approach outperforms previous work on complex table parsing datasets and leads to state-of-the-art (SOTA) performance.

Copied to clipboard

Kelvin Han, Claire Gardent

Adaptive Question Answering: Enhancing Language Model Proficiency for Addressing Knowledge Conflicts with Source Citations (2024.emnlp-main)

Challenge:	Existing methods for QG-QA are limited to English, but can be used in other languages.
Approach:	They propose to bring multilinguality to multimodal QG-QA by using Brazilian Portuguese and Russian data.
Outcome:	The proposed approach outperforms a baseline on English and can handle both languages.

Copied to clipboard

Sagi Shaier, Ari Kobren, Philip Ogren

LastingBench: Defend Benchmarks Against Knowledge Leakage (2025.findings-emnlp)

Challenge:	Existing work on citation generation has focused on unambiguous settings with single answers, failing to address the complexity of real-world scenarios.
Approach:	They propose a task of QA with source citation in ambiguous settings where multiple valid answers exist, where multiple sources exist.
Outcome:	The proposed framework generates multiple answers and cites their sources, allowing users to verify the factuality of each answer and make informed decisions.

Copied to clipboard

Yixiong Fang, Tianran Sun, Yuling Shi, Min Wang, Xiaodong Gu

KE-MHISTO: Towards a Multilingual Historical Knowledge Extraction Benchmark for Addressing the Long-Tail Problem (2025.findings-acl)

Challenge:	Existing methods to detect and safeguard LLMs against knowledge leakage fail to address the long-term challenge of mitigating it.
Approach:	They propose a method to reinforce and safeguard existing benchmarks against knowledge leakage by perturbation-based detection and counterfactual rewriting to disrupt memorization while preserving original intent.
Outcome:	The proposed method reduces memorization effects in long-context QA benchmarks, providing a more accurate assessment of model reasoning and generalization abilities.

Copied to clipboard

Arianna Graciotti, Leonardo Piano, Nicolas Lazzari, Enrico Daga, Rocco Tripodi, Valentina Presutti, Livio Pompianu

Multilinguality

Challenge:	Large Language Models struggle when probed for long-tail knowledge due to the inherent sparsity of such data.
Approach:	They propose a multilingual benchmark for Entity Linking and Question Answering in the domain of historical music knowledge that provides broader coverage of long-tail knowledge.
Outcome:	The proposed model provides broader coverage of long-tail knowledge compared to existing models.

Prompting-based Synthetic Data Generation for Few-Shot Question Answering (2024.lrec-main)

Copied to clipboard

Maximilian Schmidt, Andrea Bartezzaghi, Ngoc Thang Vu

FLARE: Faithful Logic-Aided Reasoning and Exploration (2025.emnlp-main)

Challenge:	Language models have boosted the performance of Question Answering, but data annotation is costly.
Approach:	They propose to use large language models to improve Question Answering performance . they argue that domain-agnostic knowledge from LMs is sufficient to create a well-curated dataset.
Outcome:	The proposed model outperforms state-of-the-art approaches on few-shot Question Answering.

Copied to clipboard

Erik Arakelyan, Pasquale Minervini, Patrick Lewis, Pat Verga, Isabelle Augenstein

Promptception: How Sensitive Are Large Multimodal Models to Prompts? (2025.findings-emnlp)

Challenge:	Modern Question Answering (QA) and Reasoning approaches with Large Language Models (LLMs) use Chain-of-Thought (CoT) prompting but struggle with ambiguous tasks.
Approach:	They propose a method that uses large language models to plan solutions and formalize queries without external solvers to generate outputs faithful to their intermediate reasoning chains.
Outcome:	The proposed method achieves SOTA results on 7 out of 9 diverse reasoning benchmarks and 3 out of 3 logic inference benchmarks while enabling measurement of reasoning faithfulness.

Copied to clipboard

Mohamed Insaf Ismithdeen, Muhammad Uzair Khattak, Salman Khan

Interpretable Traces, Unexpected Outcomes: Investigating the Disconnect in Trace-Based Knowledge Distillation (2026.acl-long)

Challenge:	despite the success of Large Multimodal Models, prompt design for MCQA remains poorly understood.
Approach:	They propose a framework for evaluating prompt sensitivity in LMMs . they propose 61 prompt types, each targeting specific aspects of prompt formulation .
Outcome:	The proposed framework evaluates 10 LMMs across 3 MCQA benchmarks.

Copied to clipboard

Siddhant Bhambri, Upasana Biswas, Subbarao Kambhampati

LiteraryQA: Towards Effective Evaluation of Long-document Narrative QA (2025.emnlp-main)

Challenge:	Recent advances in reasoning-oriented Large Language Models have been driven by the introduction of Chain-of-Thought (CoT) traces.
Approach:	They propose to use CoT traces to guide model inference and serve as supervision signals for Knowledge Distillation to improve smaller models.
Outcome:	The proposed model is based on a rule-based problem decomposition method and is valid for both semantic correctness and interpretability to the end user.

Copied to clipboard

Tommaso Bonomo, Luca Gioffré, Roberto Navigli

Retrieving Support to Rank Answers in Open-Domain Question Answering (2025.emnlp-main)

Challenge:	Existing Question Answering systems are limited by noisy documents and flawed QA pairs.
Approach:	They propose a high-quality subset of NarrativeQA focused on literary works . they identify and correct low-quality QA samples while removing extraneous text .
Outcome:	The proposed subset of NarrativeQA is based on literary works.

Copied to clipboard

Zeyu Zhang, Alessandro Moschitti, Thuy Vu

Beyond Static Synthetic Noise: Assessing the Robustness of Large Language Models to Natural Context Variation in the Real World (2026.findings-acl)

Challenge:	a novel question answering architecture retrieves content relevant to the combined pair . previous work on automatic claim verification has shown hallucinations .
Approach:	They propose a question-answer architecture that prioritizes supporting evidence . it retrieves paragraphs that directly substantiate the correctness of a with respect to q .
Outcome:	The proposed approach can be used by large language models to retrieve explanatory paragraphs that ground their reasoning.

Copied to clipboard

Yulong Wu, Viktor Schlegel, Riza Batista-Navarro